Selector-Enhancer: Learning Dynamic Selection of Local and Non-local Attention Operation for Speech Enhancement

Attention mechanisms, such as local and non-local attention, play a fundamental role in recent deep learning based speech enhancement (SE) systems. However, natural contains many fast-changing relatively briefly acoustic events, therefore, capturing the most informative features by indiscriminately using attention is challenged. We observe that noise type feature vary within sequence of can respectively process different types corrupted regions. To leverage this, we propose Selector-Enhancer, dual-attention convolution neural network (CNN) with feature-filter dynamically select regions from low-resolution feed them to or operations. In particular, proposed trained reinforcement (RL) developed difficulty-regulated reward related performance, model complexity “the difficulty SE task”. The results show our method achieves comparable superior performance existing approaches. Selector-Enhancer effective for real-world denoising, where number are varies on single noisy mixture.